Appratus for encoding image, apparatus for decoding image and image sensor

ABSTRACT

An image encoding apparatus includes a compressor that generates a bitstream including encoded data corresponding to values of original pixels, based on one of encoding modes, and a reconstructor that generates values of reference pixels by reconstructing the bitstream. In a first mode of the encoding modes, the compressor generates the bitstream based on a difference value between each of the values of the original pixels and a reference value which is based on at least one of the values of the reference pixels. In a second mode of the encoding modes, the compressor generates the bitstream based on an average value of at least two of the values of the original pixels.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of and claims the benefit ofpriority under 35 U.S.C. §§ 120 to U.S. patent application Ser. No.16/288,815, filed on Feb. 28, 2019, which claimed the benefits ofpriority under 35 U.S.C. § 119 to Korean Patent Application No.10-2018-0063230 filed on Jun. 1, 2018, in the Korean IntellectualProperty Office (KIPO), the entire disclosure of each of which areincorporated by reference herein in their entireties.

BACKGROUND

Various example embodiments of the inventive concepts described hereinrelate to an electronic apparatus, system, and/or method, and moreparticularly, relate to an apparatus, system, and/or method for encodingan image and an apparatus, system, and/or method for decoding an image.

Image encoding and/or image compression may refer to the process ofgenerating encoded image data, the size of which is smaller than that ofthe original image data, from the original image data. Also, imagedecoding and/or image decompression may refer to the process ofgenerating reconstructed image data by decoding the encoded image dataand/or a bitstream. The reconstructed image data may be the same as, ordifferent from, the original image data depending on the encoding anddecoding scheme.

Nowadays, as dual cameras are mounted on various electronic apparatusesand/or the number of images which may be captured per second increases,the size of image data which are stored in the electronic apparatuses isincreasing.

SUMMARY

Various example embodiments of the inventive concepts provide anapparatus, system, and/or method for encoding an original image of aBayer pattern, an apparatus, system, and/or method for decoding theencoded image, and/or an image sensor.

The technical problems to be solved by various example embodiments ofthe inventive concepts are not limited to the above-described technicalproblems, and other technical problems can be deduced from the followingexample embodiments.

According to at least one example embodiment, an image encodingapparatus may include a at least one processor configured to receive oneor more original pixels, generate a compressed bitstream includingencoded data corresponding to values of the one or more original pixels,based on an encoding mode selected from a plurality of encoding modes,the generating including generating the compressed bitstream based on adifference value between each of the values of the original pixels and areference value, the reference value is based on at least one of thevalues of one or more previously encoded reference pixels, in responseto the encoding mode being a first mode of the plurality of encodingmodes, and generating the compressed bitstream based on an average valueof at least two of the values of the original pixels in response to theencoding mode being a second mode of the plurality of encoding modes,and generate values of one or more current reference pixels byreconstructing the compressed bitstream.

According to at least one example embodiment, an image decodingapparatus may include at least one processor configured to decode areceived bitstream based on an encoding mode selected from a pluralityof encoding modes, generate values of one or more reconstruction pixelscorresponding to values of one or more original pixels based on resultsof the decoding, the generating including, decoding the bitstream basedon a difference value included in the bitstream and a reference value,the reference value is based on at least one value of one or morepreviously encoded reference pixels, in response to the encoding modebeing a first mode of the plurality of encoding modes, decoding thebitstream based on an average value received from the bitstream inresponse to the encoding mode being a second mode of the plurality ofencoding modes, and the difference value indicates a difference betweeneach of the values of the one or more original pixels and the referencevalue, and the average value is a value obtained by averaging at leasttwo of the values of the one or more original pixels, and generatevalues of one or more current reference pixels from the values of theone or more reconstruction pixels.

According to at least one example embodiment, an image sensor connectedto an memory may include at least one processor configured to generateencoded data corresponding to values of one or more original pixels of aBayer color pattern based on an encoding mode selected from a pluralityof encoding modes, generate values of one or more reconstruction pixelscorresponding to the values of the one or more original pixels bydecoding the encoded data output from the memory, the generatingincluding generating the encoded data based on a difference valuebetween each of the values of the one or more original pixels and areference value, the reference value is based on at least one of thevalues of one or more previously received original pixels referencepixels, in response to the encoding mode being a first mode of theplurality of encoding modes, and generating the encoded data based on anaverage value of at least two of the values of the original pixels inresponse to the encoding mode being a second mode of the plurality ofencoding modes, and a memory controller configured to input and outputthe encoded data to and from the memory.

BRIEF DESCRIPTION OF THE FIGURES

The above and other objects and features of the inventive concepts willbecome apparent by describing in detail various example embodimentsthereof with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

FIG. 2 is a diagram illustrating an image of a Bayer pattern obtainedthrough a Bayer color filter, according to at least one exampleembodiment.

FIG. 3 is a block diagram illustrating an encoder according to at leastone example embodiment.

FIG. 4 is a diagram illustrating an operation in which an encoderencodes pieces of original pixel data based on a DPCM (DifferentialPulse Code Modulation) mode according to at least one exampleembodiment.

FIG. 5A is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on an average mode,according to at least one example embodiment.

FIG. 5B is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on an average mode,according to at least one other example embodiment.

FIG. 5C is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on an average mode,according to at least one other example embodiment.

FIG. 6 is a table illustrating information about encoding modes to beused in an image processing apparatus of FIG. 1, according to at leastone example embodiment.

FIG. 7 is a flowchart illustrating a method in which an encoder of FIG.3 determines an encoding mode for pieces of original pixel data amongencoding modes illustrated in FIG. 6, according to at least one exampleembodiment.

FIG. 8 is a block diagram illustrating a decoder according to at leastone example embodiment.

FIG. 9 is a diagram illustrating an operation in which a decoder of FIG.8 decodes pieces of original pixel data based on a DPCM mode, accordingto at least one example embodiment.

FIG. 10 is a diagram illustrating an operation in which a decoder ofFIG. 8 decodes pieces of original pixel data based on an average mode,according to at least one example embodiment.

FIG. 11 is a diagram illustrating an operation in which a decoder ofFIG. 8 decodes pieces of original pixel data based on an average mode,according to at least one other example embodiment.

FIG. 12 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

FIG. 13 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

DETAILED DESCRIPTION

Below, various example embodiments of the inventive concepts will bedescribed in detail and clearly to such an extent that those(hereinafter referred to as “ordinary those”) skilled in the art mayimplement one or more of the inventive concepts.

FIG. 1 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment. FIG. 2 is a diagramillustrating an image of a Bayer pattern obtained through a Bayer colorfilter, according to at least one example embodiment.

An image processing apparatus 1000 is an electronic apparatus which maycapture and/or store an image associated with at least one subject byusing a solid state image sensor such as a complementary metal-oxidesemiconductor (CMOS) image sensor, etc. The image processing apparatus1000 may be a digital camera, a digital camcorder, a mobile phone,and/or a tablet computer, etc., but is not limited thereto.

Referring to FIG. 1, the image processing apparatus 1000 may include aBayer color filter 1200, an encoder 1400, a decoder 1600, and/or anapplication processor 1800, etc., but is not limited thereto. Modulesillustrated in FIG. 1 may be implemented with one processor or aplurality of processors, one or more processors including a plurality ofprocessing cores, application specific processing devices (e.g., ASICs,FPGAs, etc.), and/or may be independently implemented with a pluralityof different types of processors.

The Bayer color filter 1200 may obtain pieces of original pixel datafrom a light signal. The pieces of original pixel data may include apixel value of an original pixel. A Bayer pattern depends on thecondition that an eye of a person deduces most of lumiance data from agreen component of a subject. For example, 50% of pixels included in theBayer color filter 1200 may detect a green signal, 25% of the pixels maydetect a red signal, and 25% of the pixels may detect a blue signal.According to at least one example embodiment, the Bayer color filter1200 may have a configuration in which, for example, 2-by-2 cellpatterns each including a red (R) pixel, a blue (B) pixel, and two green(G) pixels are repeatedly arranged, but the example embodiments are notlimited thereto, for example the cell patterns may be a different sizedcell pattern. According to another example embodiment, the Bayer colorfilter 1200 may have a configuration in which, for example, 2-by-2 cellpatterns each including a red pixel, a blue pixel, and two wide green(W) pixels is repeatedly arranged, but the example embodiments are notlimited thereto.

The encoder 1400 may compress pieces of original pixel data obtainedthrough the Bayer color filter 1200, thereby reducing the size of imagedata. Referring to FIG. 2, a Bayer image 2000 indicates pieces oforiginal pixel data of a Bayer pattern obtained through the Bayer colorfilter 1200 according to at least one example embodiment. According toat least one example embodiment, the encoder 1400 may generate encodeddata associated with the pieces of original pixel data 2200 by usingpieces of reference pixel data 2400. Additionally, the encoder 1400 maygenerate the encoded data by using an average value of the pieces oforiginal pixel data 2200 without using the pieces of reference pixeldata 2400.

The encoder 1400 may determine pieces of original pixel data, which aresuccessive as much as the reference number, as one encoding unit (and/orchannel), like the pieces of original pixel data 2200 of FIG. 2. Piecesof original pixel data included in one encoding unit may be encoded to abitstream which is composed of a mode region for storing informationabout an encoding mode and/or a pixel region for storing encoded dataassociated with a value of original pixels, but is not limited thereto.

An operation of the encoder 1400 will be more fully described withreference to FIGS. 3 to 7 according to some example embodiments.

Returning to FIG. 1, the decoder 1600 may receive encoded data (i.e., abitstream) generated by the encoder 1400 and may generate decoded databy performing at least one decoding operation on the received encodingdata. The decoder 1600 may transfer the decoded data to the applicationprocessor 1800, but is not limited thereto. An operation of the decoder1600 will be more fully described with reference to FIGS. 8 to 11.

The application processor 1800 may be a central processing unit (CPU), amicroprocessor, and/or a micro controller unit (MCU), that has beenspecially configured and/or is specially programmed to perform postprocessing on the decoded data. The application processor 1800 mayperform post processing on pieces of decoded data received from thedecoder 1600. The post processing may mean applying an image enhancementalgorithm to image artifacts, etc., but is not limited thereto. Forexample, the application processor 1800 may perform, but is not limitedto, white balancing, denoising, demosaicking, lens shading, and/or gammacorrections, etc., on the received decoded data.

Below, the term “pixel” or “pixel value” may mean information and/or avalue output and/or obtained from a light signal through pixel elementsconstituting the Bayer color filter 1200. Below, the term “originalpixels” may mean pixel values of unit original pixels to be currentlyencoded and/or decoded. Below, the term “reference pixels” may meanpixel values of pixels to be referenced for the encoding of originalpixels. Below, the term “reconstruction pixels” may mean pixel values ofreconstruction pixels generated by decoding encoded data associated withoriginal pixels.

FIG. 3 is a block diagram illustrating an encoder according to at leastone example embodiment.

An encoder 3000 may output a bitstream including encoded data bycompressing received original pixels. The encoder 3000 corresponds tothe at least one example embodiment of the encoder 1400 of FIG. 1.Accordingly, even though omitted below, the above description given withregard to the encoder 1400 of FIG. 1 may be applied to the encoder 3000of FIG. 3.

Referring to FIG. 3, the encoder 3000 may include a bad pixel detector3200, a compressor 3400, a reconstructor 3600, and/or a reference pixelbuffer 3800, but the example embodiments are not limited thereto. Theencoder 3000 may include a central processor (not illustrated) and/orother special purpose processor which controls the bad pixel detector3200, the compressor 3400, the reconstructor 3600, and/or the referencepixel buffer 3800, etc., overall. Additionally, each of the bad pixeldetector 3200, the compressor 3400, the reconstructor 3600, and/or thereference pixel buffer 3800, etc., may be operated by its own processor(not illustrated), and the encoder 3000 may be overall operated as theprocessors (not illustrated) operate mutually and/or in a parallel ordistributed manner. Additionally, the bad pixel detector 3200, thecompressor 3400, the reconstructor 3600, and/or the reference pixelbuffer 3800, etc., may be controlled under control of an externalprocessor (not illustrated) of the encoder 3000.

The Bad pixel detector 3200 may detect a bad pixel which is present inone or more pieces of pixel data obtained by the Bayer color filter1200. For example, the bad pixel may include a static bad pixel causedby a physical error existing at a specific location of the Bayer colorfilter 1200 and/or a dynamic bad pixel that was caused irregularlyand/or intermittently. According to at least one example embodiment, thebad pixel detector 3200 may compare signal levels of a plurality ofpixels arranged in a horizontal direction with respect to a pixel to betested and may determine whether the pixel targeted for a test operationis included in an edge region of the whole image; in the case where thepixel to be tested is not included in the edge region, the bad pixeldetector 3200 may determine whether the pixel to be tested is defective,or in other words, whether the pixel value output by a correspondingpixel of the Bayer color filter is inaccurate and/or producing improperpixel values. According to at least one example embodiment, a bad pixelmay be detected by comparing signal levels of surrounding pixels of thetarget pixel. The bad pixel detector 3200 may tag attribute information(e.g., a flag, metadata, etc.) indicating a bad pixel to pixels whichare determined as a bad pixel.

The compressor 3400 may perform encoding (e.g., compression) on originalpixels, but is not limited thereto. According to at least one exampleembodiment, original pixels received by the compressor 3400 may bepixels which are marked by the bad pixel detector 3200 so as to indicatewhether original pixels are bad pixels (e.g., pixels that). Thecompressor 3400 may generate a bitstream (e.g., a compressed bitstream)including encoded data of an original pixel based on one of a pluralityof encoding modes. The encoding modes may include a differential pulsecode modulation (DPCM) mode in which encoding is performed based on adifference value between a value of each of the original pixels and areference value determined from one reference pixel; and an average modein which encoding is performed based on an average value of values ofthe original pixels. The reference pixel used in the DPCM mode may referto a previously encoded original pixel, and information about thereference pixel may be read from the reference pixel buffer 3800. Theencoding operation of the compressor 3400 will be more fully describedwith reference to FIGS. 4 to 7.

According to at least one example embodiment, the reconstructor 3600 maygenerate reference pixels by reconstructing the encoded data output fromthe compressor 3400. The expression “to refer to a previously encodedpixel” may mean to use a decoded pixel generated from an encodedoriginal pixel, and not to use the original pixel itself as a referencepixel. With regard to the original pixel, information about an originalpixel and information about an encoded pixel and a decoded pixel from anoriginal pixel may be present in the encoder 3000, but a decoder (e.g.,1600 of FIG. 1, etc.) does not/cannot know information about theoriginal pixel. That is, in the case where the encoder 3000 uses aprevious original pixel as a reference pixel, since the decoder cannotknow information about the previous original pixel referenced by theencoder 3000, the results of reconstruction of the original pixel mayvary, and a mismatch may occur between the encoder 3000 and the decoder.Accordingly, the encoder 3000 should use a pixel, which is againreconstructed from previously encoded original pixels, as a referencepixel.

The reconstructor 3600 may store the reconstructed pixel to a memory(not illustrated). The memory (not illustrated) may include a volatilememory such as a dynamic random access memory (DRAM) and/or a static RAM(SRAM), etc., and/or a nonvolatile memory such as a phase-change randomaccess memory (PRAM), a magnetic random access memory (MRAM), aresistive RAM (ReRAM), and/or a ferroelectric RAM (FRAM), etc.

The reference pixel buffer 3800 may extract information about at leastone reference pixel for encoding of current original pixels from thememory (not illustrated). A reference pixel may be neighborhood pixelsof original pixels, or in other words, the one or more reference pixelsmay be one or pixels neighboring the one or more original pixels.According to at least one example embodiment, the reference pixel buffer3800 may be composed of line memories (e.g., lines of memories) forstoring pixel values of neighborhood pixels of original pixels desiredand/or necessary for encoding of the original pixels. The referencepixel buffer 3800 according to at least one example embodiment may beimplemented with, but is not limited to, a volatile memory such as aDRAM and/or an SRAM.

Below, operations of the encoder 3000 will be more fully described withreference to FIGS. 4 to 7C. Operations to be described with reference toFIGS. 4 to 7C will be performed by the compressor 3400, but is notlimited thereto.

FIG. 4 is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on a DPCM modeaccording to at least one example embodiment.

In the DPCM mode, the encoder 3000 may encode original pixels 4200 basedon a reference value determined from at least one reference pixel, butis not limited thereto. According to at least one example embodiment,pixels positioned on, for example, the upper two lines of the originalpixels 4200 may be used as reference pixels, however the exampleembodiments are not limited thereto, and other pixels may be used as thereference pixels. The original pixels 4200 may include a first originalpixel G_(T0) being a green pixel, a second original pixel R_(T0) being ared pixel, a third original pixel G_(T1) being a green pixel, and afourth original pixel R_(T1) being a red pixel, but are not limitedthereto. According to another example embodiment, the second originalpixel R_(T0) and the fourth original pixel R_(T1) may be a blue pixel,etc. However, for convenience of description, it is assumed that thesecond original pixel R_(T0) and the fourth original pixel R_(T1) are ared pixel. The original pixels 4200 may be pixels before compression,but are not limited thereto. For example, a pixel value of each of theoriginal pixels 4200 may be expressed by a value which is equal to orgreater than “0” and is smaller than “1024”.

Referring to a bitstream 4800 generated by the encoder 3000 by encodingthe original pixels 4200, the bitstream 4800 may include a mode regionfor storing information (e.g., a bit string indicating the DPCM mode,etc.) about an encoding mode of the encoder 3000 used to generate thebitstream 4800, and pixel regions, e.g., DPCM1 to DMCM4, etc., forstoring information about a pixel value related to pixels correspondingto the original pixels 4200. For example, the bitstream 4800 may becomposed of a first pixel region DPCM1 for storing encoded dataassociated with the first original pixel G_(T0), a second pixel regionDPCM2 for storing encoded data associated with the second original pixelR_(T0), a third pixel region DPCM3 for storing encoded data associatedwith the third original pixel G_(T1), and a fourth pixel region DPCM4for storing encoded data associated with the fourth original pixelR_(T1), etc.

The mode region Mode may include M-bit data. Each of the first pixelregion DPCM1, the second pixel region DPCM2, the third pixel regionDPCM3, and the fourth pixel region DPCM4 may include N-bit data. “M” and“N” may be positive integers which are greater than “0”. Below, it isassumed that “M” and “N” are “4”, but the example embodiments are notlimited thereto.

The encoder 3000 may store a difference value d1 between a pixel valueof the first original pixel G_(T0) and a reference value to the firstpixel region DPCM1. According to at least one example embodiment, anaverage value of a pixel value of a reference pixel G0 positioned on afirst location, e.g., the upper left side of the first original pixelG_(T0), and a pixel value of a reference pixel G1 positioned on a secondlocation, e.g., the upper right side thereof, may be used as a referencevalue, but the example embodiments are not limited thereto. That is, thedifference value d1 stored to the first pixel region DPCM1 may bedetermined by the following Equation 1.

d1=(G0+G1)/2−G _(T0)  [Equation 1]

The encoder 3000 may store a difference value d3 between a pixel valueof the second original pixel R_(T0) and a reference value to the secondpixel region DPCM2. According to at least one example embodiment, apixel value of a reference pixel RB1 positioned on, e.g., the secondline, above the second original pixel R_(T0) may be used as a referencevalue. That is, the difference value d3 stored to the second pixelregion DPCM2 may be determined by the following Equation 2.

d3=RB1−R _(T0)  [Equation 2]

The encoder 3000 may store a difference value d2 between a pixel valueof the third original pixel G_(T1) and a reference value. According toat least one example embodiment, an average value between a pixel valueof a reference pixel G1 positioned on a first location, e.g., the upperleft side of the third original pixel G_(T1), and a pixel value of areference pixel G2 positioned on a second location, e.g., the upperright side of the third original pixel G_(T1), may be used as areference value. That is, the difference value d2 stored to the thirdpixel region DPCM3 may be determined by the following Equation 3.

d2=(G1+G2)/2−G _(T1)  [Equation 3]

The encoder 3000 may store a difference value d4 between a pixel valueof the fourth original pixel R_(T1) and a reference value to the fourthpixel region DPCM4. According to at least one example embodiment, apixel value of a reference pixel RB2 positioned on the second line abovethe fourth original pixel R_(T1) may be used as a reference value. Thatis, the difference value d4 stored to the fourth pixel region DPCM4 maybe determined by the following Equation 4.

d4=RB2−R _(T1)  [Equation 4]

Since each of the difference values, e.g., d1, d2, d3, and d4, etc.,determined by Equation 1 to Equation 4 may be a negative value or apositive value, the most significant bit of each of the pixel regionsDPCM1, DPCM2, DPCM3, and DPCM4 may be a sign bit indicating signinformation. For example, in the case where the difference value d1determined by Equation 1 is “−6”, a bit string of “1110” including asign bit of “1” indicating a negative value and a bit string of “110”indicating “6” may be stored to the first pixel region DPCM1.

Additionally, the encoder 3000 according to at least one exampleembodiment may perform a bit shift operation on each of the differencevalues d1, d2, d3, and d4, etc., determined by Equation 1 to Equation 4based on the size of a relevant pixel region. For example, in the casewhere the number of bits assigned to the first pixel region DPCM1 is “4”and the difference value d1 is 1610(10000₂), the encoder 3000 may storea bit string of “100”, which is obtained by performing a right bit shiftoperation (“>>”) on the difference value d1 two times, to the firstpixel region DPCM1 together with the signal bit of “0”. The encoder 3000may store information about the number of times that a shift operationis performed, to the mode region.

However, positions of pixels to be referenced for encoding of originalpixels described with reference to FIG. 4 may be only one exampleembodiment and may be changed. For example, the encoder 3000 may not usereference pixels determined as a bad pixel by the bad pixel detector3200 of FIG. 3 for the purpose of determining a reference value.

In the case of encoding the first original pixel G_(T0), in the casewhere the reference pixel G0 is a bad pixel, the encoder 3000 maydetermine not an average value of the reference pixels G0 and G1, butrather a pixel value of the reference pixel G1 as the reference value.In the case where the reference pixels G0 and G1 are a bad pixel, apixel value of a reference pixel G5 may be determined as a referencevalue. In the case where reference pixels G0, G1, and G5 are bad pixels,reference values may be determined based on at least one pixel value ofreference pixels G4 and G6 positioned in a diagonal direction, etc.

In the case of encoding the second original pixel R_(T0), for example,in the case where the reference pixel RB1 is a bad pixel, the encoder3000 may sort reference pixels RB0, RB2, and RB3, etc., to select anintermediate value and may determine the selected intermediate value asa reference value.

FIG. 5A is a diagram illustrating an operation in which an encoder ofFIG. 3 encodes pieces of original pixel data based on a first averagemode, according to at least one example embodiment.

The encoder 3000 may encode original pixels 5220 based on a firstaverage mode and may generate a bitstream 5300, but is not limitedthereto.

Referring to the bitstream 5300, the bitstream 5300 may include a moderegion 5340 including information (e.g., a bit string indicating thefirst average mode, etc.) about an encoding mode, a first pixel region5360 for storing encoded data associated with the first original pixelG_(T0) and the third original pixel G_(T1) being a green pixel, and asecond pixel region 5380 for storing encoded data associated with thesecond original pixel R_(T0) and the fourth original pixel R_(T1) beinga red pixel, but the example embodiments are not limited thereto.

The encoder 3000 may store an average value of a pixel value of thefirst original pixel G_(T0) and a pixel value of the third originalpixel G_(T1) to the first pixel region 5360. The encoder 3000 mayperform a bit shift operation on the first average value depending on(and/or based on) the size of the first pixel region 5360. For example,in the case where the number of bits assigned to the first pixel region5360 is “8” and the first average value is 48810(=1111010002), theencoder 3000 may perform a right bit shift operation (“>>”) on the firstaverage value and may store “1111010” to the first pixel region 5360. Inthis case, since the pieces of encoded data associated with two originalpixels G_(T0) and G_(T1) are the same as each other, the encoded data ofthe first and third original pixels G_(T0) and G_(T1) may be storedtogether in the first pixel region 5360.

The encoder 3000 may store a second average value of a pixel value ofthe second original pixel R_(T0) and a pixel value of the fourthoriginal pixel R_(T1) to the second pixel region 5380. A bit shiftoperation may be performed on the second average value depending on thesize of the second pixel region 5380, identically to the way to storethe first average value to the first pixel region 5360.

The first average mode described with reference to FIG. 5A may beefficient when the upper side of the original pixels 5220 corresponds toa horizontal edge. In this case, since spatial correlation almost doesnot exist between reference pixels positioned on the upper side of theoriginal pixels 5220, it may be more efficient to perform encoding byusing the pixel values of the original pixels 5220 itself.

FIGS. 5B and 5C are diagrams illustrating an operation in which anencoder of FIG. 3 encodes pieces of original pixel data based on anaverage mode, according to another example embodiment.

FIG. 5B shows an operation in which the encoder 3000 encodes originalpixels 5420 based on a second average mode and generates a bitstream5500 according to at least one example embodiment, and FIG. 5C shows anoperation in which the encoder 3000 encodes original pixels 5620 basedon a third average mode and generates a bitstream 5700 according to atleast one example embodiment. The encoding based on the second averagemode and the third average mode may be especially efficient when ahorizontal edge exists at the upper side of original pixels and a pixelincluded in an edge region is also present in the original pixels.

Referring to FIG. 5B, the fourth original pixel R_(T1) of the originalpixels 5420 may be a pixel included in an edge region. A bitstream 5500may include a mode region 5520 including information (e.g., a bit stringindicating the second average mode, etc.) about an encoding mode, afirst pixel region 5540 for storing encoded data associated with thefirst original pixel G_(T0) and the third original pixel G_(T1), asecond pixel region 5560 for storing encoded data associated with thesecond original pixel R_(T0), and a third pixel region 5580 for storingencoded data associated with the fourth original pixel R_(T1), but theexample embodiments are not limited thereto.

The encoder 3000 may store an average value of a pixel value of thefirst original pixel G_(T0) and a pixel value of the third originalpixel G_(T1) to the first pixel region 5540. However, since the fourthoriginal pixel R_(T1) corresponds to an edge component, spatialcorrelation almost does not exist between the second original pixelR_(T0) and the fourth original pixel R_(T1). Accordingly, in the casewhere an average value of the second original pixel R_(T0) and thefourth original pixel R_(T1) is used as encoded data, the informationloss may be great. Accordingly, the encoder 3000 may respectively storea pixel value of the second original pixel R_(T0) and a pixel value ofthe fourth original pixel R_(T1) to the second pixel region 5560 and thethird pixel region 5580 without separate conversion (e.g., a bit shiftoperation which is performed depending on the size of the second pixelregion 5560 and the third pixel region 5580 is excluded).

Referring to FIG. 5C, the first original pixel G_(T0) of the originalpixels 5620 may correspond to an edge component according to at leastone example embodiment. A bitstream 5700 may include a mode region 5720including information (e.g., a bit string indicating the third averagemode, etc.) about an encoding mode, a first pixel region 5740 forstoring encoded data associated with the first original pixel G_(T0), asecond pixel region 5760 for storing encoded data associated with thethird original pixel G_(T1), and a third pixel region 5780 for storingencoded data associated with the second original pixel R_(T0) and thefourth original pixel R_(T1).

A method for encoding the original pixels 5620 based on the thirdaverage mode is similar to the method for encoding the original pixels5420 based on the second average mode, which is described with referenceto FIG. 5B. The difference is as follows. Since an edge component of theoriginal pixels 5620 is the first original pixel G_(T0), a pixel valueof the first original pixel G_(T0) and a pixel value of the thirdoriginal pixel G_(T1) may be respectively stored to the first pixelregion 5740 and the second pixel region 5760 without separate conversion(e.g., a bit shift operation which is performed depending on the size ofthe first pixel region 5740 and the second pixel region 5760 isexcluded). For example, the encoder 3000 may store an average value of apixel value of the second original pixel R_(T0) and a pixel value of thefourth original pixel R_(T1) to the third pixel region 5780.

According to at least one example embodiment, the encoder 3000 mayhandle the second average mode described with reference to FIG. 5B andthe third average mode described with reference to FIG. 5C as the sameencoding mode and may include an additional bit for distinguishing thesecond average mode from the third average mode in the generatedbitstream. For example, the mode region 5520 of the bitstream 5500 andthe mode region 5720 of the bitstream 5700 may include an additionalregion for indicating sub mode information, but is not limited thereto.The encoder 3000 may distinguish the second average mode from the thirdaverage mode by recording a value, e.g., of “0” or “1” in the additionalregion. In this case, the size of each of pixel regions of the bitstream5500 and the bitstream 5700 may decrease due to the additional regionfor indicating the sub mode information.

FIG. 6 is a table illustrating information about encoding modes to beused in an image processing apparatus of FIG. 1, according to at leastone example embodiment.

Referring to a table 6000, encoding modes may include the DPCM modedescribed with reference to FIG. 4, the average mode described withreference to FIGS. 5A to 5C, and/or a pulse code modulation (PCM) mode,but the example embodiments are not limited thereto. For example, 13encoding modes, e.g., Mode 0 to Mode 12, are stored in a first columnMode of the table 6000, but the example embodiments are not limitedthereto. Encoding methods DPCM, Average, and/or PCM, etc., which areused in the 13 encoding modes are stored in a second column Method butthe example embodiments are not limited thereto. The number of timesthat a bit shift operation is performed to store a difference valueand/or an average value associated with a pixel value, e.g., a greenpixel, is stated at a third column Shift G. The number of times that abit shift operation is performed to store a difference value and/or anaverage value associated with a red or blue pixel is recorded at afourth column Shift R/B, etc.

The table 6000 defines a protocol between the encoder 3000 and thedecoder 1600. That is, the table 6000 indicates information desiredand/or necessary for the decoder 1600 to generate a reconstruction pixelbased on a bitstream received from the encoder 1400. Pieces ofinformation recorded in the table 6000 may be experimentally obtainedvalues.

For example, the encoder 1400 may compress original pixels based on Mode4 of the table 6000 and may record “4” in a mode region of the generatedbitstream. The decoder 1600 may parse “4” from the mode region of thebitstream received from the encoder 1400. The decoder 1600 may determinethat the current original pixels are compressed in the DPCM mode basedon the values stored in Method column (e.g., category) of the table6000, and the decoder 1600 may determine that a difference value of agreen pixel stored in each of a first pixel region and a third pixelregion is a result value obtained by performing a bit shift operationthree times based on the Shift G column (e.g., category) of the table6000. Also, the decoder 1600 may determine that a difference value of ared/blue pixel stored in each of a second pixel region and a fourthpixel region is a result value obtained by performing a bit shiftoperation three times based on the Shift R/B column (e.g., category) ofthe table 6000.

For example, the encoder 1400 may compress the original pixels based onMode 10 of the table 6000 and may record “10” in a mode region of thegenerated bitstream. The decoder 1600 may determine that currentoriginal pixels are compressed in the average mode and an average valueof green pixels stored in the first pixel region is a result valueobtained by performing a bit shift operation two times from the table6000. Also, the decoder 1600 may determine that an average value of ared/blue pixel stored in the second pixel region is a result valueobtained by performing a bit shift operation two times.

According to at least one example embodiment, Mode 12 of the table 6000may indicate the PCM mode, but the example embodiments are not limitedthereto. The PCM mode may be a mode of including only the upper bitvalues in a bitstream suitably for the size of a relevant pixel region,without sequentially converting the pixel values of current originalpixels. The encoder 1400 according to at least one example embodimentmay attempt encoding by using Mode 0 to Mode 12 sequentially, and mayperform encoding based on the PCM mode when Mode 1 to Mode 12 all fail,but the example embodiments are not limited thereto.

FIG. 7 is a flowchart illustrating a method in which an encoder of FIG.3 determines an encoding mode for pieces of original pixel data amongencoding modes illustrated in FIG. 6, according to at least one exampleembodiment.

In operation S7100, the encoder 3000 may determine whether to encodeoriginal pixels based on the DPCM mode. For example, the encoder 3000may sequentially determine whether to encode original pixels based onvarious modes corresponding to DPCM, e.g., Mode 1 to Mode 9 in the table6000 of FIG. 6. That is, in the case where encoding is possible in acurrent mode (Yes), the encoder 3000 may determine a current DPCM mode“Mode n” (e.g., n being any one of 1 to 9) as an encoding mode; if not(No), the encoder 3000 may determine whether encoding is possible in anext DPCM mode. Since the number of times of a bit shift operationincreases when progressing toward Mode 9 from Mode 1, the loss oforiginal data due to compression may occur less when encoding isperformed in a mode having a smaller number (e.g., compression usingMode 1 may result in less loss/error than compression of original datausing Mode 9, etc.).

For example, the encoder 3000 may determine whether to encode theoriginal pixels based on Mode 0 in the table 6000 of FIG. 6. Referringto the table 6000, a bit shift operation is not performed in Mode 0.Accordingly, in the case where any one of the difference value d1associated with the first original pixel G_(T0), the difference value d2associated with the second original pixel R_(T0), the difference valued3 associated with the third original pixel G_(T1), and the differencevalue d4 associated with the fourth original pixel R_(T1), is notexpressed within a given bit set (e.g., four bits) set to the size of apixel region, the encoder 3000 may determine that it is difficult,undesirable, and/or impossible to encode the original pixels using Mode0. That is, in the case where it is determined that Mode 0 is usable,the encoder 3000 may determine Mode 0 as an encoding mode for theoriginal pixels; however, in the case where it is determined that Mode 0is unusable, the encoder 3000 may determine whether to use a next DPCMmode (i.e., Mode 1).

In the case where it is determined based on the DPCM mode that encodingof the original pixels is difficult, undesirable, and/or impossible(No), in operation S7300, the encoder 3000 may determine whether toencode the original pixels based on the average mode. For example, theencoder 3000 may sequentially determine whether to encode the originalpixels currently based on, e.g., Mode 10 and Mode 11, in the table 6000of FIG. 6. In the case where it is determined that encoding is currentlypossible in the average mode “Mode k” (e.g., k being 10 or 11, etc.), inoperation S7500, the encoder 3000 may perform an additional testoperation. However, in the case where encoding is difficult,undesirable, and/or impossible even though any mode of Mode 10 and Mode11 is used (No), in operation S7400, the encoder 3000 may determine thePCM mode as an encoding mode for the original pixels.

In operation S7500, the encoder 3000 may compare the degree of data lossdue to compression of the average mode “Mode k”, which is determined inoperation S73000 as encoding may be possible, and the degree of dataloss due to compression of the PCM mode. As described above, a bit shiftoperation may be performed depending on the size of a pixel regionassigned to a bitstream, and the encoder 3000 may compare the levels ofdata loss by comparing the number of times that a bit shift operation isperformed in the average mode “Mode k” and the number of times that abit shift operation is performed in the PCM mode. In the case where thelevel of data loss in the PCM mode is greater than the level of dataloss in the average mode “Mode k” determined in operation S7300 (Yes),the encoder 3000 may determine the average mode “Mode k” determined inoperation S7300 as an encoding mode for the original pixels (S7600). Inthe case where the level of data loss in the PCM mode is not greaterthan the level of data loss in the average mode Mode k determined inoperation S7300 (No), the encoder 3000 may determine the PCM mode as anencoding mode for the original pixels (S7400).

FIG. 8 is a block diagram illustrating a decoder according to at leastone example embodiment.

A decoder 8000 corresponds to a detailed example embodiment of thedecoder 1600 of FIG. 1, but is not limited thereto. Accordingly, eventhough omitted below, the above description given with regard to thedecoder 1600 of FIG. 1 may be applied to the decoder 8000 of FIG. 8,etc. It may be easily understood to ordinary those that an operationcorresponding to the encoder 3000 of FIG. 3 may be performed in thedecoder 8000.

Referring to FIG. 8, the decoder 8000 may include a bad pixel detector8200, a decompressor 8400, a reconstructor 8600, and/or a referencepixel buffer 8800, but is not limited thereto. The decoder 8000 mayinclude at least one central processor (not illustrated) which controlsthe bad pixel detector 8200, the decompressor 8400, the reconstructor8600, and/or the reference pixel buffer 8800, etc. Additionally, each ofthe bad pixel detector 8200, the decompressor 8400, the reconstructor8600, and/or the reference pixel buffer 8800 may be operated by theirown processor (not illustrated), and the decoder 8000 may be overalloperated as the processors (not illustrated) operate mutually and/or ina parallel or distributed manner. Additionally, the bad pixel detector8200, the decompressor 8400, the reconstructor 8600, and/or thereference pixel buffer 8800 may be controlled under the control of oneor more external processors (not illustrated) of the decoder 8000.

The bad pixel detector 8200, the reconstructor 8600, and/or thereference pixel buffer 8800 correspond to the bad pixel detector 3200,the reconstructor 3600, and/or the reference pixel buffer 3800 of FIG.3, and thus, additional description will be omitted to avoid redundancy.

The decompressor 8400 may determine an encoding mode from a bitstreamreceived from the encoder 3000 of FIG. 3, and may generate pieces ofreconstruction pixel data based on the determined encoding mode, but isnot limited thereto. According to at least one example embodiment, thedecompressor 8400 may parse information included in a mode region of thereceived bitstream and may determine an encoding mode for originalpixels based on the parsed information, but is not limited thereto.

Below, operations of the decoder 8000 will be more fully described withreference to FIGS. 9 to 11 according to at least one example embodiment.Operations to be described with reference to FIGS. 9 to 11 will beperformed by the decompressor 8400 according to at least one exampleembodiment.

FIG. 9 is a diagram illustrating an operation in which a decoder of FIG.8 decodes original pixels based on a DPCM mode, according to at leastone example embodiment.

The decoder 8000 may determine an encoding mode for original pixels asthe DPCM mode with reference to a mode region Mode of a receivedbitstream 9000. For example, it is assumed that an encoding mode isdetermined as Mode 1 of FIG. 6, but is not limited thereto.

The decoder 8000 may generate a first reconstruction pixel G′_(R0)associated with, for example, the first original pixel G_(R0) by using afirst difference value d1 read from the first pixel region DPCM1 and areference value. In the DPCM mode, since a green pixel is encoded byusing, as a reference value, an average value of a pixel value of areference pixel positioned on a first location, e.g., the upper leftside of an original pixel, and a pixel value of a reference pixelpositioned on a second location, e.g., the upper right side thereof, thedecoder 8000 may read information about the reference pixel from thereference pixel buffer 8800 and may determine the reference value basedon the read reference pixel information corresponding to the firstlocation and the second location. That is, the decoder 8000 maydetermine a value of the first reconstruction pixel G′_(R0) by using thefollowing Equation 5.

G′R0=(G0+G1)/2−d1(G0 and G1:reference pixel)  [Equation 5]

The decoder 8000 may generate a reconstruction pixel R′_(R0) associatedwith the second original pixel R_(R0) by using the second differencevalue d2 read from the second pixel region DPCM2 and a reference value.In the DPCM mode, since a red or blue pixel is encoded by using a pixelvalue of a reference pixel positioned on, e.g., an upper two lines ofthe original pixel as a reference value, the decoder 8000 may readinformation about the reference pixel from the reference pixel buffer8800 and may determine the reference value based on the read referencepixel information. That is, the decoder 8000 may determine a value ofthe second reconstruction pixel R′_(R0) by using the following Equation6.

R′R0=RB1−d2(RB1:reference pixel)  [Equation 6]

The decoder 8000 may respectively read the third difference value d3 andthe fourth difference value d4 from the third pixel region DPCM3 and thefourth pixel region DPCM4 and may generate a third reconstruction pixelG′_(R1) and a fourth reconstruction pixel R′_(R1) by using the readvalues and a reference value. A method in which the decoder 8000generates the third reconstruction pixel G′_(R1) and the fourthreconstruction pixel R′_(R1) is the same as the method in which thedecoder 8000 generates the first reconstruction pixel G′_(R0) and thesecond reconstruction pixel R′_(R0), and thus, additional descriptionwill be omitted to avoid redundancy.

According to at least one example embodiment, the decoder 8000 mayperform a bit shift operation, in an opposite direction to a bit shiftoperation which the encoder 3000 performs a bit shift operation on eachof the difference values d1, d2, d3, and d4, etc., read from pixelregions.

For example, in the case where an encoding mode is determined as Mode 3of FIG. 6, the decoder 8000 may perform a left bit shift operation(“<<”) (e.g., the opposite direction bit shift operation than the bitshift operation performed by the encoder) on the first difference valued1 read from the first pixel region DPCM1 and the third difference valued3 read from the third pixel region DPCM3 two times, and may generatethe first reconstruction pixel G′_(R0) and the third reconstructionpixel G′_(R1) based on result values of the left bit shift operation.The decoder 8000 may perform a left bit shift operation (“<<”) on thesecond difference value d2 read from the second pixel region DPCM2 andthe fourth difference value d4 read from the fourth pixel region DPCM4three times and may generate the second reconstruction pixel R′_(R0) andthe fourth reconstruction pixel R′_(R1) based on result values of theleft bit shift operation.

FIG. 10 is a diagram illustrating an operation in which a decoder ofFIG. 8 decodes original pixels based on an average mode, according to atleast one example embodiment.

The decoder 8000 may determine an encoding mode for original pixels asthe average mode with reference to a mode region 10020 of a receivedbitstream 10000, but is not limited thereto.

The decoder 8000 may determine a first average value d1 read from afirst pixel region 10040 as a pixel value of the first reconstructionpixel G′_(R0) and the third reconstruction pixel G′_(R1).

The decoder 8000 may determine a second average value d2 read from asecond pixel region 10060 as a pixel value of the second reconstructionpixel R′_(R0) and the fourth reconstruction pixel R′_(R1).

As in the DPCM mode, the decoder 8000 according to at least one exampleembodiment may perform a left bit shift operation (“<<”) on each of thefirst average value d1 read from the first pixel region 10040 and thesecond average value d2 read from the second pixel region 10060, basedon information read from the mode region 10020.

For example, in the case where an encoding mode is determined as Mode 10of FIG. 6, the decoder 8000 may determine a value, which is obtained byperforming a left bit shift operation (“<<”) on the first average valued1 two times, as a pixel value of the first reconstruction pixel G′_(R0)and the third reconstruction pixel G′_(R1), and may determine a value,which is obtained by performing a left bit shift operation (“<<”) on thesecond average value d2 two times, as a pixel value of the secondreconstruction pixel R′_(R0) and the fourth reconstruction pixelR′_(R1).

FIG. 11 is a diagram illustrating an operation in which a decoder ofFIG. 8 decodes original pixels based on an average mode, according toanother example embodiment.

The decoder 8000 may determine an encoding mode for original pixels asthe average mode with reference to a mode region 11020 of a receivedbitstream 11000. For example, it is assumed that an encoding mode isdetermined as Mode 11 of FIG. 6, but the example embodiments are notlimited thereto. According to at least one example embodiment, in thecase where an additional region Sub mode is included in the mode region11020 including mode information, the decoder 8000 may determine anencoding mode as Mode 11, etc.

As described with reference to FIGS. 5B and 5C, a decoding method may bedifferently determined depending (e.g., based) on a value read from theadditional region Sub mode. For example, in the case where a value readfrom the additional region Sub mode is “0” may indicate that an edgecomponent is present in a right pixel of original pixels, and the casewhere a value read from the additional region Sub mode is “1” mayindicate that an edge component is present in a left pixel of theoriginal pixels, but is not limited thereto.

According to at least one example embodiment, in the case where a valueread from the additional region Sub mode is “0”, depending on the numberof times of a bit shift operation defined in the table 6000 of FIG. 6,the decoder 8000 may determine a value, which is obtained by performinga left bit shift operation (“<<”) on a value d1 read from a first pixelregion 11040 five times, as a pixel value of the first reconstructionpixel G′_(R0) and a pixel value of the third reconstruction pixelG′_(R1). The decoder 8000 may determine a value, which is obtained byperforming a left bit shift operation (“<<”) on a value d2 read from asecond pixel region 11060 five times, as a pixel value of the secondreconstruction pixel R′_(R0). Also, the decoder 8000 may determine avalue, which is obtained by performing a left bit shift operation (“<<”)on a value d3 read from a third pixel region 11080 five times, as apixel value of the second reconstruction pixel R′_(R1).

According to at least one example embodiment, in the case where a valueread from the additional region Sub mode is “1”, depending on the numberof times of a bit shift operation defined in the table 6000 of FIG. 6,the decoder 8000 may determine a value, which is obtained by performinga left bit shift operation (“<<”) on the value d1 read from the firstpixel region 11040 five times, as a pixel value of the firstreconstruction pixel G′_(R0). The decoder 8000 may determine a value,which is obtained by performing a left bit shift operation (“<<”) on thevalue d2 read from the second pixel region 11060 five times, as a pixelvalue of the third reconstruction pixel G′_(R1). Also, the decoder 8000may determine a value, which is obtained by performing a left bit shiftoperation (“<<”) on the value d3 read from the third pixel region 11080five times, as a pixel value of the second reconstruction pixel R′_(R0)and a pixel value of the fourth reconstruction pixel R′_(R1).

FIG. 12 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

An image processing apparatus 12000 corresponds to a detailed exampleembodiment of the image processing apparatus 1000 of FIG. 1, but is notlimited thereto. Accordingly, even though omitted below, the abovedescription given with regard to the image processing apparatus 1000 ofFIG. 1 may be applied to the image processing apparatus 12000.

Referring to FIG. 12, the image processing apparatus 12000 may includean image sensor 12200, a memory 12400, and/or an application processor12600, etc., but the example embodiments are not limited thereto.

The image sensor 12200 may be an electronic part which extracts colorinformation from a light signal so as to be implemented in a display(e.g., a display panel, a display device, a projector, etc.). The imagesensor 12200 may include an encoder 12220, a decoder 12240, and/or amemory controller 12260, etc., but is not limited thereto. The imagesensor 12200 is also called a “3 stack sensor”.

The encoder 12220 may perform the operation of the encoder 1400described with reference to FIGS. 1 to 7 according to at least oneexample embodiment. That is, the encoder 12220 may encode originalpixels of a received Bayer pattern based on, for example, the DPCM modeand the average mode, etc., and may generate a bitstream includingencoded data. The bitstream generated by the encoder 12220 may beprovided to the memory controller 12260.

The memory controller 12260 may control an operation of inputting andoutputting the encoded data. The bitstream generated by the encoder12220 may be input to the memory 12400 under the control of the memorycontroller 12260. The memory controller 12260 may include a dedicatedlogic circuit (e.g., FPGA or ASICs, etc.) which performs variousoperations for controlling overall operations in the memory 12400.

The memory 12400 may be connected to the image sensor 12200 and maystore image frames. The memory 12400 may store not original dataassociated with image frames but pieces of encoded data generated by theencoder 12220.

Accordingly, the number of image frames which may be stored to thememory 12400 may greatly increase compared to the case of storingoriginal data to the memory 12400. The memory 12400 may include avolatile memory such as a DRAM and/or an SRAM, etc., and/or anonvolatile memory such as a flash memory, a PRAM, an MRAM, a ReRAM,and/or an FRAM, etc.

The memory 12400 may output a bitstream including encoded data to thedecoder 12240 under the control of the memory controller 12260. Thedecoder 12240 may perform the operation of the decoder 1600 describedwith reference to FIGS. 1 and 8 to 12 according to at least one exampleembodiment. That is, the decoder 12240 may read information about anencoding mode from the bitstream received from the memory controller12260 and may generate reconstruction image data based on the readencoding mode.

The application processor 12600 may apply various image processingtechniques based on image data received from the image sensor 12200. Theapplication processor 12600 may correspond to the application processor1800 of FIG. 1, but is not limited thereto, and thus, additionaldescription will be omitted to avoid redundancy.

FIG. 13 is a block diagram illustrating an image processing apparatusaccording to at least one example embodiment.

An image processing apparatus 13000 corresponds to at least one exampleembodiment, such as the image processing apparatus 1000 of FIG. 1.Accordingly, even though omitted below, the above description given withregard to the image processing apparatus 1000 of FIG. 1 may be appliedto the image processing apparatus 13000.

Referring to FIG. 13, the image processing apparatus 13000 may includean image sensor 13200, an MIPI (Mobile Industry Processor Interface)13400, and/or an application processor 13600, etc., but the exampleembodiments are not limited thereto. The image sensor 13200 is alsocalled a “2 stack sensor”.

The image sensor 13200 may include an image signal processor (ISP) 13220and an encoder 13240. The ISP 13220 may perform pre-processing onoriginal image data of a Bayer pattern. The pre-processing may meanapplying various image improvement algorithms which may improve thequality of image data. The pre-processed image data may be provided tothe encoder 13240.

The encoder 13240 may encode original image data of a Bayer patternreceived from the ISP 13220 and may output a bitstream including encodeddata. The encoder 13240 may perform the operation of the encoder 1400described with reference to FIGS. 1 to 7 according to at least oneexample embodiment, but is not limited thereto. That is, the encoder13240 may encode received original pixels based on the DPCM mode and theaverage mode and may generate a bitstream, etc. The bitstream generatedby the encoder 13240 may be provided to the MIPI 13400.

The MIPI 13400 which is an intermediate interface between the imagesensor 13200 and the application processor 13600 may be an electronicpart implemented with a circuit which may transmit an electrical signal.The MIPI 13400 may perform an interface operation for transmitting thereceived bitstream to the application processor 13600. The MIPI 13400may provide the bitstream received from the encoder 13240 to theapplication processor 13600. Accordingly, the image processing apparatus13000 may greatly increase the number of image frames which aretransmitted between the image sensor 13200 and the application processor13600 per second, by transmitting encoded data instead of transmittingoriginal data associated with image frames from the image sensor 13200to the application processor 13600.

The application processor 13600 may receive the bitstream including theencoded data from the MIPI 13400. The application processor 13600 mayinclude a decoder 13620 for decoding the received bitstream. That is,the decoder 13620 may read information about an encoding mode from thebitstream received from the MIPI 13400 and may generate reconstructionimage data based on the read encoding mode. The application processor13600 may apply various image processing techniques to thereconstruction image data.

An image processing apparatus according to at least one exampleembodiment of the inventive concepts may perform compression on originalBayer image. As the size of image data decreases through thecompression, efficiency associated with a memory space and a bandwidthof the image processing apparatus may be improved.

While the inventive concepts has been described with reference tovarious example embodiments thereof, it will be apparent to those ofordinary skill in the art that various changes and modifications may bemade thereto without departing from the spirit and scope of the exampleembodiments of the inventive concepts as set forth in the followingclaims.

What is claimed is:
 1. An image encoding apparatus comprising: at leastone processor configured to, receive one or more original pixels; andgenerate a compressed bitstream including encoded data corresponding tovalues of the one or more original pixels, based on an encoding modeselected from a plurality of encoding modes, the generating thecompressed bitstream further including, generating the compressedbitstream based on a difference value between each of the values of theoriginal pixels and a reference value, the reference value being basedon at least one of the values of one or more reference pixels previouslyencoded and reconstructed, in response to the encoding mode being afirst mode of the plurality of encoding modes, generating the compressedbitstream based on an average value of at least two of the values of theoriginal pixels in response to the encoding mode being a second mode ofthe plurality of encoding modes, and the compressed bitstream includes apixel region storing the difference value in the first mode and theaverage value in the second mode.
 2. The image encoding apparatus ofclaim 1, wherein the at least one processor is further configured to:perform a bit shift operation on the difference value or the averagevalue based on a size of the pixel region; and determine one of thefirst mode and the second mode as the encoding mode for the values ofthe one or more original pixels, and wherein the compressed bitstreamincludes a mode region for storing information related to the determinedencoding mode and the number of times that the bit shift operation isperformed.
 3. The image encoding apparatus of claim 1, wherein the atleast one processor is further configured to: obtain the values of theone or more original pixels using a Bayer color filter, the values ofthe one or more original pixels including at least a value of a firstpixel, a value of a second pixel, a value of a third pixel, and a valueof a fourth pixel; and the first pixel and the second pixel are a greenpixel, and the third pixel and the fourth pixel are a red or blue pixel.4. The image encoding apparatus of claim 3, wherein, in the first mode,the at least one processor is configured to: store a first differencevalue between the value of the first pixel and an average value of avalue of a first reference pixel and a value of a second reference pixelin the pixel region; and store a second difference value between thevalue of the second pixel and an average value of a value of a thirdreference pixel and a value of a fourth reference pixel in the pixelregion, and wherein the first reference pixel and the second referencepixel are a green pixel positioned on an upper line of the first pixel,and the third reference pixel and the fourth reference pixel are a greenpixel positioned on an upper line of the second pixel.
 5. The imageencoding apparatus of claim 3, wherein in the first mode, the at leastone processor is configured to: store a third difference value between avalue of a fifth reference pixel and the value of the third pixel in thepixel region; and store a fourth difference value between a value of asixth reference pixel and the value of the fourth pixel in the pixelregion, and wherein the fifth reference pixel is a red or blue pixelpositioned on the second line above the third pixel, and the sixthreference pixel is a red or blue pixel positioned on the second lineabove the fourth pixel.
 6. The image encoding apparatus of claim 3,wherein the at least one processor is further configured to: detect abad pixel from the values of the one or more original pixels, andwherein, in the first mode, a value of the bad pixel is not used as thereference value.
 7. The image encoding apparatus of claim 6, wherein theat least one processor is further configured to: in response to a firstreference pixel being detected to be the bad pixel, determine a value ofa second reference pixel as the reference value; and in response to thesecond reference pixel being further detected to be the bad pixel,determine a value of a third reference pixel as the reference value, andwherein the first reference pixel and the second reference pixel are agreen pixel positioned on an upper line of the first pixel, and thethird reference pixel is a green pixel positioned on the second lineabove the first pixel.
 8. The image encoding apparatus of claim 6,wherein the at least one processor is further configured to: in responseto a fourth reference pixel being detected to be the bad pixel, sortvalues of a fifth reference pixel, a sixth reference pixel, and aseventh reference pixel in response to the detection that the fourthreference pixel is the bad pixel; and determine a second largest value,among the sorted values, as the reference value, and wherein the fourthreference pixel is a red or blue pixel positioned on the second lineabove the third pixel, the fifth reference pixel is a red or blue pixelpositioned on the second line above the fourth pixel, and the sixthreference pixel and the seventh reference pixel are a red or blue pixelpositioned on the same line as the fourth reference pixel and the fifthreference pixel.
 9. The image encoding apparatus of claim 3, wherein, inthe second mode, the at least one processor is further configured to:store an average value of the value of the first pixel and the value ofthe second pixel and an average value of the value of the third pixeland the value of the fourth pixel, in the pixel region.
 10. The imageencoding apparatus of claim 3, wherein, in the second mode, the at leastone processor is further configured to: store an average value of thevalue of the first pixel and the value of the second pixel, the value ofthe third pixel, and the value of the fourth pixel, in the pixel region.11. The image encoding apparatus of claim 3, wherein, in the secondmode, the at least one processor is further configured to: store thevalue of the first pixel, the value of the second pixel, and an averagevalue of the value of the third pixel and the value of the fourth pixel,in the pixel region.
 12. An operation method of image encoding apparatuscomprising at least one processor, the operation method comprising:receiving, by the at least one processor, one or more original pixels;and generating, by the at least one processor, a compressed bitstreamincluding encoded data corresponding to values of the one or moreoriginal pixels, based on an encoding mode selected from a plurality ofencoding modes, the generating the compressed bitstream furtherincluding, generating the compressed bitstream based on a differencevalue between each of the values of the one or more original pixels anda reference value, the reference value being based on at least one ofthe values of one or more reference pixels previously encoded andreconstructed, in response to the encoding mode being a first mode ofthe plurality of encoding modes, generating the compressed bitstreambased on an average value of at least two of the values of the originalpixels in response to the encoding mode being a second mode of theplurality of encoding modes, and the compressed bitstream includes apixel region, wherein the pixel region stores the difference value inthe first mode and the average value in the second mode.
 13. Theoperation method of claim 12, further comprising: performing, by the atleast one processor, a bit shift operation on the difference value orthe average value based on a size of the pixel region; and determining,by the at least one processor, the encoding mode for the values of theone or more original pixels, and wherein the compressed bitstreamincludes a mode region for storing information related to the determinedencoding mode and the number of times that the bit shift operation isperformed.
 14. The operation method of claim 13, wherein the determiningthe encoding mode comprises: determining whether to encode the originalpixels using the first mode based on the number of times that the bitshift operation is performed and the size of the pixel region; anddetermining whether to encode the original pixels using the second modein response to the determination not to encode the original pixels usingthe first mode.
 15. The operation method of claim 14, wherein thegenerating the compressed bitstream further includes, generating thecompressed bitstream based on a value of upper bits of the originalpixels, a number of upper bits is determined based on the size of thepixel region, in response to the encoding mode being a third mode of theplurality of encoding modes; and the determining the encoding modefurther includes, comparing the number of times that the bit shiftoperation is performed in the second mode with the number of times thatthe bit shift operation is performed in the third mode.
 16. Theoperation method of claim 12, further comprising: obtaining, by the atleast one processor, the values of the one or more original pixels usinga Bayer color filter, after the receiving the one or more originalpixels, and wherein the values of the one or more original pixelsincluding at least a value of a first pixel, a value of a second pixel,a value of a third pixel, and a value of a fourth pixel, and the firstpixel and the second pixel are a green pixel, and the third pixel andthe fourth pixel are a red or blue pixel.
 17. The operation method ofclaim 16, wherein, in the first mode, the generating the compressedbitstream further comprises: storing, in the pixel region, a firstdifference value between the value of the first pixel and an averagevalue of a value of a first reference pixel and a value of a secondreference pixel, and a second difference value between the value of thesecond pixel and an average value of a value of a third reference pixeland a value of a fourth reference pixel; and the first reference pixeland the second reference pixel are a green pixel positioned on an upperline of the first pixel, and the third reference pixel and the fourthreference pixel are a green pixel positioned on an upper line of thesecond pixel.
 18. The operation method of claim 16, wherein in the firstmode, the generating the compressed bitstream further comprises:storing, in the pixel region, a third difference value between a valueof a fifth reference pixel and the value of the third pixel, and afourth difference value between a value of a sixth reference pixel andthe value of the fourth pixel in the pixel region; and wherein the fifthreference pixel is a red or blue pixel positioned on the second lineabove the third pixel, and the sixth reference pixel is a red or bluepixel positioned on the second line above the fourth pixel.
 19. Theoperation method of claim 16, wherein in the second mode, the generatingthe compressed bitstream further comprises: storing an average value ofthe value of the first pixel and the value of the second pixel and anaverage value of the value of the third pixel and the value of thefourth pixel, in the pixel region.
 20. An image sensor connected with amemory, the image sensor comprising: an encoder configured to generateencoded data corresponding to values of original pixels of a Bayer colorpattern based on one of a plurality of encoding modes; a memorycontroller configured to control an operation of inputting andoutputting the encoded data to and from the memory; and a decoderconfigured to generate values of reconstruction pixels corresponding tothe values of the original pixels by decoding the encoded data outputfrom the memory, in a first mode of the plurality of encoding modes, theencoder is further configured to generate the encoded data based on adifference value between each of the values of the original pixels and areference value which is based on at least one of the values ofreference pixels previously encoded and reconstructed, in a second modeof the plurality of encoding modes, the encoder is further configured togenerate the encoded data based on an average value of at least two ofthe values of the original pixels, and the encoder is further configuredto store the difference value or the average value in a pixel region ofthe generated encoded data.